Network traffic measurement allows network managers to monitor network activities and to plan for network operations. Tools for measuring traffic need to scale with a wide spectrum of applications, flows and queries while maintaining the performance of the underlying hardware in terms of accuracy and speed. Conventional solutions sample traffic and send per-flow statistics to a remote server that serves user applications. Measuring per-flow traffic often results in inaccuracy and sending per-flow statistics often results in intensive resource and network bandwidth usage.
Conventional flow-based measurement techniques generally use a large number of managed hardware counters due to a direct correlation between the number of counters and the flow table size. Further, conventional techniques typically gather per-flow statistics continually in dataplane without consideration of application needs. It is up to user applications to pull desired flow counters. This disconnection between user applications and traffic measurement induces a large number of transactions, excessive bandwidth usage and may affect packet processing performances.
For example, Cisco Netflow is a data monitoring and collection system. In a Netflow-capable switch or router, flow statistics are tracked at the datapath continuously at a specific sampling rate. Statistics of each flow are sent to a Netflow Data Collector which serves user applications. The Netflow-capable switch creates a number of transactions with a bandwidth usage proportional to the number of flows regardless of the actual need of the applications.
A recent approach to reducing the number of hardware counters was proposed in ProgME: towards programmable network measurement, IEEE/ACM Transactions on Networking, February 2011, vol. 19, No. 1. The architecture proposed there is based on analyzing individual packets contents with a binary decision diagram (BDD), and finding out an optimal allocation of flows using a Disentangling algorithm. However, there is no evidence of convergence or optimality for the Disentangling algorithm especially with large flow descriptors such as in the context of OpenFlow capable switches.